As the capabilities of computing devices continue to evolve, many different ways have been introduced to allow users to interact with computing devices, such as through mechanical devices (e.g., keyboards, mice, etc.), touch screens, motion, and gesture. Many computing devices are now capable of receiving and responding to voice commands, including desktops, tablets, entertainment systems, and portable communication devices. Due to their usefulness, it is still early to know, but it might be the beginning of an emerging trend for users to place multiple voice-enabled devices throughout their homes to receive and respond to their voice commands in different locations in their homes. Having multiple voice-enabled devices placed throughout an environment increases the complexity of determining how to respond to a voice command. Provided herein are specific technical solutions for improving existing systems so they are more useful for users having multiple voice controlled devices.